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Dear  Educator: 

The  four  booklets  in  this  series  discuss  the  reading, 
mathematics,  science  and  social  studies  results  of  the  1988 
Massachusetts  Educational  Assessment  Program.   They 
represent  one  of  the  many  efforts  of  the  Department  of 
Education  to  help  schools  carry  out  their  educational 
mission  more  effectively.   In  this  case,  they  provide  models 
for  student  evaluation  within  the  classroom,  as  well  as 
describing  students'  progress  in  understanding. 

The  title  of  this  series,  On  Their  Own,  suggests  an 
important  aim  of  education:  the  ability  of  students  to  act 
as  independent,  rational  thinkers.   The  questions  described 
in  these  booklets  demand  that  ability.   They  demand  active 
intelligence  as  students  are  required  to  relate  what  they 
know  to  new  and  challenging  situations. 

In  addition  to  describing  students'  understanding,  these 
booklets  carry  a  message  about  the  evaluation  that  goes  on 
in  the  classrooms.  The  message  is  that  the  short  objective 
tests  of  facts  or  procedures,  standard  fare  in  most 
classrooms,  are  too  slight  a  vehicle  to  convey  the  true 
purpose  of  evaluation. 

In  the  first  place,  effective  student  evaluation  is  an 
important  component  of  effective  teaching.   Research  has 
described  the  complex  thinking  that  underlies  students' 
errors  and  misconceptions.  Unless  teachers  take  the  time  to 
discover  for  themselves  how  students  understand  a  subject, 
they  will  be  unable  to  adjust  their  teaching  in  appropriate 
ways.  This  kind  of  evaluation,  involving  student  discussion 
and  explanation,   should  be  a  continuous  and  constant  part 
of  every  classroom. 


Secondly,  evaluation  can,  and  does,  affect  students' 
learning.   Not  only  does  it  signal  for  the  student  the 
content  areas  that  teachers  consider  important,  it  gives  a 
message  about  the  kind  of  thinking  that  is  considered 
valuable.   When  testing  is  limited  to  short  objective 
questions,  requiring  a  single  answer,  the  message  given  is 
that  facts  are  what  really  count.   When  questions  encourage 
students  to  think,  to  grapple  with  the  material  and  to 
consolidate  different  aspects  of  learning,  the  message  is 
much  different.  Such  questions  indicate  to  students  that  it 
is  the  quality  of  thought  that  is  important,  not  the 
correctness  of  the  answer  itself.   The  possibility  of 
different  answers  opens  the  door  for  discussion, 
argumentation,  and  intellectual  excitement  in  our  schools. 
This  is  the  message  that  we  want  to  convey  to  our  students. 

We  hope  that  you  will  study  the  material  included  in  this 
series  and  incorporate  the  ideas  presented  in  your  own 
classrooms . 


Sincerely  yours, 


yj&dti* 


Harold  RayngdLds  Jr. 
Commissioner  of  Education 
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As  any  good  reader  is  aware,  reading  is  not  just  the  passive  absorption  of 
what  is  on  the  printed  page.  It  is  an  active  process  whereby  we  create, 
extend,  and  reflectively  assimilate  meaning  from  what  we  read  by  selectively 
bringing  to  bear  things  that  we  already  know  and  blending  them  with  what  we 
get  from  the  text. 

Even  so,  such  a  process  may  yield  a  superficial  understanding  of  the  text. 
Many  people  select  little  from  what  they  already  know.  They  relate  this  to 
what  they  are  reading  only  in  cursory  ways.  They  form  no  strategies  to 
enhance  their  comprehension.  To  achieve  understanding,  insight,  and  to  gain 
information  we  are  sure  is  accurate,  we  must  use  the  most  skillful  thinking  we 
can  muster  in  connecting  what  we  are  reading  to  what  we  know  and  in 
determining  its  value.  The  best  readers  do  this  with  ease.  Those  that  don't 
nonetheless  have  the  capability  of  doing  it  and  can  learn  to. 

Reading  educators  have,  of  course,  acknowledged  the  role  of  active  skillful 
thinking  in  relation  to  a  text  as  the  key  to  good  reading  for  many  years.  But 
this  has  not  always  yielded  an  approach  and  well-articulated  conception  of 
the  good  thinker.  Such  a  conception  acknowledges  the  importance  of  the 
skillful  use  of  ordinary  thinking  activities  like  predicting  and  generalizing,  and 
the  concomitant  need  to  support  interpretations  of  our  experience  and  judg- 
ments about  both  fact  and  values.  The  good  thinker  is  the  critical  and  creative 
thinker. 

Critical  and  creative  thinking  takes  on  a  special  form  in  reading  a  text.  Alert 
readers  continually  adjust  their  understanding  as  they  pick  up  clues  about  the 
type  of  text  they  are  dealing  with  (external  factors)  and  the  way  a  particular 
text  develops  or  unfolds  (internal  factors).  Good  reading  is  a  function  of  the 
way  these  clues  call  up  knowledge  beyond  the  text  and  the  way  we  actively 
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synthesize  it  with  what  we  are  reading  by  using  the  skills  of  critical  and  creative 
thought. 

The  impact  of  this  approach  on  classroom  instruction  in  reading  is  dramatic. 
In  the  new  reading  classroom,  students  engage  directly  with  whole  works  and 
not  worksheets.  The  thinking  promoted  in  these  classrooms  is  the  thinking 
we  would  expect  of  any  good  thinker.  It  involves  ordinary  processes  of 
thought  done  well,  not  just  low-level  decoding  skills.  This  is  accomplished  as 
students  read  a  work  through  the  use  of  explicit  thinking  strategies.  These 
engage  students  in  careful  thought  about  the  text  in  ways  that  deliberately 
relate  it  to  other  information  relevant  to  understanding  and  assessing  what  is 
being  read. 

In  such  thinking  classrooms  we  find  the  continued  use  of  appropriate  ques- 
tioning strategies,  metacognitive  techniques,  and  cooperative  learning  struc- 
tures. These  are  the  backbone  of  good  thinking  skills  instruction  —  they 
enhance  students'  thoughtful  reading  as  they  do  any  form  of  thinking.  The 
model  of  teaching  critical  thinking  skills  by  infusing  their  use  into  rich  content 
instruction  provides  the  reading  teacher  with  all  the  classroom  strategies 
needed  to  speak  systematically  to  this  more  holistic  approach  to  good  reading. 

Teaching  reading  well,  though,  requires  careful  monitoring  of  students' 
progress.  Good  lesson  design  must  meet  students  where  they  are.  Treating 
good  reading  as  good  thinking  creates  a  challenge  with  regard  to  assessment. 
Put  simply,  we  must  be  able  to  expose  the  way  students  think  about  what  they 
are  reading  to  assess  it. 

The  new  departure  in  state  assessment  described  in  this  booklet  provides  just 
the  right  model  for  the  classroom  teacher.  Ordinary  multiple-choice  testing, 
although  providing  us  with  some  information  about  student  progress,  is  not 
the  best  instrument  to  reveal  how  students  are  thinking.  We  can  never  find 
out  directly  why  students  make  the  choices  they  do  in  answering  multiple- 
choice  items.  Yet  this  is  exactly  what  we  want  to  know.  By  contrast,  open- 
ended  items  in  which  students  are  challenged  by  appropriate  questions  to 
perform  the  kind  of  thinking  we  are  trying  to  teach  them  —  the  focus  of  this 
booklet  —  give  us  exactly  what  we  want.  The  perceptive  analysis  of  student 
thinking  included  in  the  pages  of  this  booklet  shows  us  how  much  we  can  learn 
from  such  tests.  This  information  is  invaluable  for  any  classroom  teacher. 
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In  the  spring  of  1988,  the  Massachusetts  Department  of  Education  ad- 
ministered its  second  biennial  assessment.  It  tested  all  eligible  fourth, 
eighth,  and  twelfth  grade  students  in  four  content  areas  —  reading,  mathe- 
matics, science,  and  social  studies.  Although  the  large  majority  of  the  over 
3000  items  were  given  in  multiple-choice  format,  some  of  the  items  were 
open-ended,  requiring  students  to  answer  in  written  form.  These  open-ended 
questions  appeared  in  one  form  of  the  tests  at  each  grade  level.  Consequently, 
one-twelfth  of  the  fourth  grade  students,  one-sixteenth  of  the  eighth  grade 
students,  and  one-twentieth  of  the  twelfth  grade  students  received  a  test  form 
that  contained  some  open-ended  questions.  However,  they  did  not  all  receive 
the  same  questions.  The  ten  or  so  questions  in  each  subject  area  were 
distributed  in  such  a  way  that,  while  we  were  able  to  receive  a  sufficiently  large 
number  of  responses  to  each  question  to  report  reliably,  we  were  not  able  to 
cover  the  many  types  of  thinking  that  each  subject  required. 

Members  of  our  Advisory  Committees,  composed  of  teachers  throughout  the 
state,  reviewed  and  categorized  all  the  answers  obtained.  Their  comments 
and  instructional  suggestions  are  reflected  in  this  report. 

There  are  three  reasons  for  our  decision  to  include  open-ended  questions  in 
our  assessment  of  student  performance.  The  first  is  the  value  of  the  informa- 
tion obtained.  While  multiple-choice  items  are  efficient,  easy  to  score,  and 
objective,  they  are  a  weak  measure  of  how  students  actually  think.  Nor  can 
they  measure  students'  ability  to  generate  solutions  or  students'  approach  to 
those  ill-structured  problems  that  are  most  familiar  in  everyday  life.  Includ- 
ing these  types  of  questions  on  the  assessment  results  in  a  more  valid  estima- 
tion of  student  achievement  than  we  would  have  obtained  had  we  limited 
assessment  to  multiple-choice  items. 

Our  second  purpose  in  including  these  open-ended  questions  was  our  belief 
in  their  intrinsic  value:  they  reflect  the  kind  of  thinking  that  education  is  all 
about.  Too  often  educators  pay  lip  service  to  the  need  for  active  learning  but 
teach  and  test  students  in  ways  that  demand  passivity.  By  their  actions, 
schools  say  to  students,  "We  are  not  interested  in  your  response;  we  are  only 
interested  in  the  correct  response."  This  report  of  the  open-ended  testing 
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shows  how  students  respond  when  they  are  given  the  opportunity  to  answer 
to  a  semi-structured  situation. 

Finally,  we  hope  that  such  testing  will  act  as  a  model  for  classroom  testing. 
We  have  given  the  questions  themselves,  as  well  as  the  state-wide  results,  in 
order  for  teachers  to  try  them  out  in  their  own  classes  and,  if  they  desire,  to 
compare  the  results  they  obtain  with  the  state  norm.  We  wish  to  show  that 
this  type  of  testing  yields  important  information  about  students'  under- 
standing of  concepts  and  procedures,  their  ability  to  apply  their  learning  to 
new  situations,  and  their  need  for  further  instruction. 

Although,  the  results  in  each  subject  area  are  treated  separately,  the  under- 
lying thought  processes  which  we  report  on  are  similar.  They  reflect  an 
approach  to  thinking  that  stresses  engagement  and  critical  evaluation. 
Beyond  this,  however,  we  look  at  how  students  function  in  various  learning 
contexts  which  require  the  understanding  of  specific  concepts. 


Of  the  four  subjects  that  are  considered  in  this  series,  reading  is  the  most 
general  and  most  fundamental.  In  a  certain  sense,  it  forms  the  gateway 
to  the  other  subjects.  Indeed,  some  have  contended  that  the  same  criteria  that 
distinguish  good  reading  from  poor  reading  distinguish  good  critical  thinking 
from  poor  thinking.  It  has  been  argued  that  the  skills  in  reading  involve  much 
more  than  a  passive  attempt  to  "understand  the  text."  Rather,  the  com- 
prehension of  a  written  passage  requires  much  the  same  skills  that  are  called 
upon  to  analyze  and  evaluate  any  kind  of  discourse,  whatever  the  form  or 
subject  matter. 

Reading  and  Thinking:  A  New  Framework  for  Comprehension  (Massachusetts 
Educational  Assessment  Program,  May  1987)  called  attention  to  these  inter- 
active and  dynamic  theories  of  reading.  Independently  of  what  a  text  says 
directly,  according  to  these  theories,  readers  bring  their  own  prior  knowledge 
and  reading  purposes  to  bear.  After  reading  several  science  fiction  stories, 
for  example,  they  start  a  new  one  with  different  expectations,  and  presumably 
better  knowledge  of  the  form,  than  if  they  had  never  encountered  science 
fiction.  Different  purposes  such  as  schoolwork  vs.  entertainment  and  dif- 
ferent techniques  such  as  scanning  vs.  close  reading  dictate  different  "styles" 
of  reading  almost  as  distinct  as  styles  of  writing.  Moreover,  while  they  must 
comprehend  the  information  actually  set  forth  in  a  text  —  "the  words  in  front 
of  them"  —  readers  frequently  extend  the  meaning  by  drawing  inferences  of 
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their  own,  on  the  text  and  on  the  subject.  In  that  sense  interactive  theorists 
say  that  reader  and  text  (or  reader  and  author)  contribute  to  a  mutual 
construction  of  meaning. 

Responses  to  the  open-ended  questions  confirm  the  instructional  recommen- 
dations contained  in  Reading  and  Thinking.  Those  recommendations  can  be 
summarized  as  follows: 

•  Give  students  the  confidence  to  form  and  apply  different  reading  strategies 
according  to  the  characteristics  of  the  material,  their  experience  with 
similar  material,  and  their  purpose  for  reading.  Encourage  them  to  activate 
prior  knowledge  and  to  keep  several  strategies  "in  stock."  To  students  used 
to  reading  everything  along  one  dimension,  this  will  be  a  revelation.  Be 
sure  they  learn  that  flexibility  is  part  of  the  method,  the  capacity  to  adjust 
or  even  change  strategies  with  the  twists  and  turns  of  a  text. 

•  Since  the  aim  is  to  broaden  skills  beyond  what  is  required  for  "doing  a 
school  lesson,"  use  materials  that  they  would  be  likely  to  encounter  outside 
reading  class.  At  the  same  time,  coach  them  in  applying  a  variety  of 
perspectives  to  even  a  single  familiar  text. 

•  Stress  inference  as  essential  to  comprehension.  Skill  in  this  area  means 
recognizing  clues  provided  by  the  author,  as  when  a  storyteller  puts  a  joke 
in  the  mouth  of  a  character  to  show,  not  tell,  that  the  character  is  witty.  For 
active  readers,  skill  also  encompasses  drawing  inferences  of  their  own.  To 
explore  the  possibility  of  bias  in  a  text,  for  example,  they  need  not  depend 
on  a  teacher  to  raise  the  issue  or  an  author  to  admit  to  bias. 

Characterizing  interactive  theory  in  broad  strokes  is  one  thing.  Translating 
it  into  daily  lessons  is  another.  Perhaps  the  flavor  can  be  conveyed  by 
contrasting  two  ways  of  setting  up  a  simple  assignment.  Suppose  a  poem  is 
handed  out  and  students  are  asked  to  answer  a  short  series  of  multiple-choice 
questions  emphasizing  recognition  of  metaphor.  In  effect,  the  teacher  has 
told  the  students  what  to  read  for,  though  not  why.  By  contrast,  an  open- 
ended  question  about  the  poet's  view  of  the  world  might  nudge  them  to 
discover  metaphor  for  themselves  and  in  fact  ask  the  teacher  to  tell  them  more 
about  it.  Open-ended  questions  may  also  uncover  gaps  of  which  the  teacher 
would  otherwise  be  unaware:  for  instance,  that  several  students  are  too 
inexperienced  with  typographical  conventions  to  distinguish  a  title  unless  it 
is  printed  in  huge  letters  or  that  they  have  never  thought  about  the  function 
of  a  title  in  a  text.  They  misread  it  as  the  first  line  of  the  poem.  In  many  cases 
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interactive  questions  can  be  framed  to  elicit  more  than  one  "right"  answer. 
If  several  plausible  interpretations  emerge  in  class,  the  stage  may  be  set  for  a 
research  assignment. 

We  have  found  that  open-ended  questions  are  highly  useful  for  assessing 
comprehension.  Furthermore,  we  believe  that  regular  inclusion  of  open- 
ended  questions  in  the  reading  curriculum  helps  to  improve  comprehension. 
Since  appropriate  questioning  is  a  major  educational  concern,  the  remainder 
of  this  booklet  is  devoted  to  actual  open-ended  questions  that  have  been  tried 
on  students.  By  analyzing  response  patterns  we  hope  to  show  how  a  critical 
thinking  approach  to  reading  comprehension  works  out  in  practice. 

Each  of  the  questions  described  in  this  booklet  taps  one  or  more  of  the 
competencies  listed  in  the  Reading  Assessment  Framework  in  the  Appendix. 
We  suggest  that  you  use  them  as  models  for  measuring  the  specific  skills  that 
your  own  students  bring  to  the  reading  process. 


A  Question  of  Answers 
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Scientists 


Your  teacher  tells  your  class,  "Now  you  are  going  to  read  about  scien- 
tists." Make  a  list  of  eight  (8)  things  you  already  know  about  scientists. 


This  question  suggests  a  possible  strategy  for  teachers  to  use  in  activating 
prior  knowledge  before  beginning  a  reading  assignment.  With  the  ac- 
knowledgment of  the  role  of  prior  knowledge  in  reading  comprehension 
comes  the  realization  that  children  often  lack  important  concepts  that  are 
helpful,  or  even  essential,  to  understanding.  Even  those  with  broader  ex- 
perience are  often  apt  to  think  in  stereotypical  ways.  Non-judgmental 
"brainstorming"  fosters  creative  thinking,  as  the  most  common  associations 
are  made  and  children  reach  out  for  more  creative  responses.  By  initially 
requiring  written  responses,  the  teacher  gives  quieter  or  more  reflective 
children  a  chance  to  contribute  without  fear  of  being  dominated  by  their  more 
articulate  classmates. 

Once  this  task  has  been  accomplished,  the  teacher  can  use  the  results  as  a 
basis  for  class  discussion,  at  the  same  time  widening  children's  concepts. 
Clustering  or  categorizing  responses  is  one  method  for  accomplishing  this. 
For  example,  after  listing  student  contributions,  the  teacher  might  ask,  "How 
many  of  these  items  are  similar?"  so  that  children  can  recognize  similar  ideas 
in  different  forms.  "How  many  different  clusters  do  you  have  in  your  list?" 
encourages  broader  thinking  and  discourages  the  tendency  to  give  formula 
answers  such  as  they  study  fossils,  they  study  rocks,  they  study  minerals. 

Finally,  this  type  of  question  is  an  example  of  a  type  of  informal  evaluation 
that  teachers  might  use  to  track  their  students'  quality  of  thinking.  Instruc- 
tional activities  that  stress  broadening  concepts  should  result  in  less  reliance 
on  formula  and  increased  fluency  in  ideas.  By  occasionally  asking  children  to 
write  down  their  ideas,  teachers  can  check  the  effectiveness  of  their  instruc- 
tion. 
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Pictured  below  is  the  top  of  a  page  from  a  science  book.    Use  this 
information  to  answer  questions  1  and  2. 


1.  Judging  from  the  title,  what  would  a  person  learn  from  reading  this 
article? 


2.  Judging  from  the  diagram,  what  else  would  a  person  learn  about 
teeth? 


A  Question  of  Answers 


Teeth 

The  simple  "toothy"  exercise  on  the  facing  page,  administered  to  children  in 
Grade  4,  called  upon  students  to  explore  the  relationship  between  form  and 
meaning.  As  a  more  structured  exercise  might  have  done,  it  encouraged 
attention  to  mechanics:  the  function  of  a  title  and  reading  graphics.  Ap- 
proximately three-fourths  of  the  respondents  gave  adequate  answers.  "Types 
of  teeth"  was  a  common  reply  to  question  1;  "different  parts  of  a  tooth"  on 
question  2. 

The  open  format,  however,  also  uncovered  a  tendency  among  a  significant 
number  of  children  which  might  otherwise  have  gone  unnoticed.  About  25 
percent  of  the  respondents  failed  to  activate  their  prior  knowledge  critically. 
In  other  words,  they  answered  questions  they  had  expected,  perhaps  from 
previous  reading,  rather  than  responding  to  the  reading  matter  actually 
before  them. 

Examples: 

How  to  floss  your  teeth  without  making  you  bleed. 

That  if  you  don't  brush  it  you  teeth  will  look  bad  and  feel  bad. 

How  to  be  a  dentist. 


On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


When  I  Read 

I'm  a  runner,  a  racer, 
I've  got  a  lot  of  speed. 
I  can  sprint  from  here  to        there 
with  time  to  spare. 
But  when  I  read 
then  I'm  a  diver! 
I  plunge 
right 
in 
and  until  the  story's  over 
I  don't  come  up  for  air. 
Then  too  I'm  an  explorer, 
a  tracker  and  a  rover 
and  I  always 
find  something 
I  didn't  know  was  there. 

Lillian  Morrison 


1 .  In  the  poem  above,  the  poet  uses  language  playfully.  What  are  two 
examples  of  this,  and  what  does  the  poet  accomplish  in  each  case? 


2.  Based  on  what  the  poem  says,  describe  several  things  the  poet  might 
do  when  reading  a  novel.  Please  support  your  ideas  with  evidence 
from  the  poem. 
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A  Question  of  Answers 


When  I  Read 


Assessment  of  the  response  of  Grade  8  students  to  Lillian  Morrison's  poem 
"When  I  Read"  —  reproduced  here  —  offers  some  interesting  contrasts 
between  multiple-choice  exercises  and  open-ended  exercises. 

The  first  question  was  used  to  measure  the  role  of  structure  in  conveying 
meaning.  At  this  level  students  may  not  be  accustomed  to  considering  how 
the  parts  of  a  poem  contribute  to  over-all  meaning.  Approximately  half  the 
respondents  were  able  to  recognize  playful  language  and  explain  its  function. 
Thus  they  noticed  the  space  deliberately  left  between  words  in  line  4,  as  well 
as  the  "downward"  typography  of  lines  8-10. 

This  was  roughly  comparable  to. success  achieved  in  a  parallel  multiple-choice 
question: 

The  space  between  the  words  "to"  and  "there"  in  line  4  of  the  poem 
"When  I  Read"  is  used  to  suggest  how 


57% 

a.  far  the  runner  ran 

7% 

b.  tired  the  runner  is 

4% 

c.  lonely  the  runner  is 

31% 

d.  fast  the  runner  is 

However,  the  two  modes  of  questioning  call  upon  different  thought  processes. 
In  one  case  the  student  is  directed  to  a  poetic  device  and  is  given  a  fixed  choice 
of  interpretations.  In  the  other  case  the  student  is  asked  to  perform  in  the 
manner  of  an  active  reader:  finding  the  device  and  analyzing  its  intent  on  his 
or  her  own. 

Among  the  children  who  were  unable  to  answer  the  open-ended  question 
satisfactorily,  20  percent  ignored  its  requirements  and  proceeded  to  sum- 
marize the  poem  as  a  whole.  Since  interpretation  plays  a  large  role  in 
classroom  discussion  of  poetry,  children  may  have  answered  a  question  they 
were  used  to  rather  than  the  question  presented. 

The  second  question  requires  interpretation  but,  unlike  many  of  this  type,  it 
does  not  ask  students  to  define  meaning.  Rather,  it  measures  students'  ability 
to  draw  inferences  about  a  speaker's  character  from  a  text. 

About  a  third  of  the  respondents  provided  satisfactory  answers.  Among 
comments  representative  of  this  group: 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


She  probably  doesn't  stop  until  the  story  is  over.  This  is  why  she 
says  she  is  a  diver  and  doesn't  come  up  for  air.  She  also  searches 
for  a  meaning  or  moral  of  the  story  she  is  reading.  That  is  why  she 
says  she  is  a  tracker  and  rover. 


Twenty  percent  were  able  to  discuss  some  parts  of  the  poem  adequately,  while 
ignoring  or  misinterpreting  the  other  parts.  For  example,  some  students 
merged  the  title  with  the  first  sentence  of  the  poem,  thereby  misinterpreting 
the  meaning  of  the  first  part. 

She  reads  fast  because  in  the  first  two  sentences  she  talks  about 
being  a  runner  and  a  racer  and  lots  of  speed. 


However,  the  question  also  provoked  awkward  attempts  at  literal  interpreta- 
tion by  20  percent  of  the  respondents.  "The  poet  might  like  to  run  a  lot  and 
relax  in  a  swimming  pool  or  something  while  reading  a  novel"  was  one  of  the 
more  surreal  ventures  in  this  direction.  Over  a  quarter  either  gave  an 
irrelevant  response  (such  as  quoting  from  the  poem  with  no  explanation)  or 
left  the  space  blank. 

Whether  a  more  structured  format  helps  students  to  do  an  abstract  task  better 
or  simply  does  part  of  the  work  for  them  remains  unclear.  A  multiple-choice 
item  produced  the  following  results: 

The  speaker  in  the  poem  "When  I  Read"  likes  reading  because  it 

20%  is  fast  like  running 

1 2%  is  a  solitary  activity 

62%  leads  to  interesting  discoveries 

6%  isn't  competitive  like  running 
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A  Question  of  Answers 


"The  Cave"  and  "The  Dreamer"  both  describe  boys  who  have  special 
places  they  think  of  as  their  own.  Read  the  poems  to  discover  how  the 
boys  and  their  special  places  are  similar. 


The  Cave 


The  Dreamer 


Sometimes  when  the  boy  was  troubled  he  would  go 
To  a  little  cave  of  stone  above  the  brook 

And  build  a  fire  just  big  enough  to  glow 
Upon  the  ledge  outside,  then  sit  and  look. 

Below  him  was  the  winding  silver  trail 
Of  water  from  the  upland  pasture  springs, 
And  meadows  where  he  heard  the  calling  quail; 
Before  him  was  the  sky,  and  passing  wings. 

The  tang  of  willow  twigs  he  lighted  there, 
Fragrance  of  meadows  breathing  slow  and  deep, 

The  cave's  own  musky  coolness  on  the  air, 
The  scent  of  sunlight ...  all  were  his  to  keep. 

We  had  such  places — cave  or  tree  or  hill . . . 
And  we  are  lucky  if  we  keep  them  still. 


He  spent  his  childhood  hours  in  a  den 
of  rushes,  watching  the  gray  rain  braille 
the  surface  of  the  river.  Concealed 
from  the  outside  world,  nestled  within, 
he  was  safe  from  parents,  God,  and  eyes 
that  looked  upon  him  accusingly, 
as  though  to  say:  Even  at  your  age, 
you  could  do  better.  His  camouflage 
was  scant,  but  it  served,  and  at  evening, 
when  fireflies  burned  holes  into  heaven, 
he  took  the  path  homeward  in  the  dark, 
a  small  Noah,  leaving  his  safe  Ark. 


William  Childress 

Copyright  1966  by  William  Childress 


Glenn  W.  Dresbach 


From  Collected  Poems  by  Glenn  W.  Dresbach, 
The  Caxton  Printers,  Ltd.  Caldwell,  Idaho. 


1 .  Both  boys  go  to  their  special  places  when  they  are  troubled.  Which 
boy  seems  to  be  more  unhappy?  Please  explain  the  reasons  for  your 
choice. 

2.  Which  poem  helped  you  to  imagine  the  boy's  special  place  better? 
Explain  how  the  poet  helped  you  to  imagine.  Include  one  example 
of  the  poet's  effort  to  help  you  imagine. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


The  Cave  /  The  Dreamer 

Two  poems  about  troubled  youth  were  set  side  by  side  in  the  next  exercise. 
In  contrast  to  the  metaphoric  style  in  the  previous  example,  these  poems 
relied  more  upon  description.  Grade  8  students  did  very  well  with  two 
open-ended  questions.  Over  80  percent  were  able  to  use  supporting  evidence 
from  the  poems  to  answer  appropriately.  This  may  reflect  familiarity  both 
with  the  type  of  poems  and  the  requirements  of  the  tasks.  Apparently,  by 
eighth  grade  students  are  comfortable  with  recognizing,  understanding,  and 
using  description  in  interpretation. 

As  to  which  boy  seemed  more  unhappy,  "I  think  The  Dreamer  is  more 
troubled"  was  a  characteristic  answer,  "because  he  thinks  nobody  thinks  he's 
good  enough.  He  also  just  sits  there,  unlike  The  Cave  where  the  boy  admires 
the  beauty  and  builds  a  fire." 

When  asked  to  imagine  each  boy's  special  place,  the  majority  preferred  the 
cave.  "This  author,"  wrote  one  student,  "used  many  more  adjectives  to 
describe  this  special  place.  The  poet's  description  also  appealed  not  only  to 
our  sight,  but  our  hearing,  smell,  and  feeling...." 

Interestingly,  a  high  number  of  students  in  the  group  that  favored  the  other 
boy's  special  place  were  able  to  make  a  reasonable  argument  for  their 
preference.  uThe  Dreamer  was  easier  to  imagine,"  said  one  of  these. 
"'Watching  the  gray  rain  braille  /  the  surface  of  the  river.'  It  gives  you  a  dark, 
gray,  sorrowful  feeling." 

Students  produced  less  well  when  presented  with  the  following  multiple- 
choice  question: 

William  Childress  titles  his  poem  "The  Dreamer"  because: 

35%  the  boy's  special  place  only  exists  in  his  dreams 

6%  when  the  boy  is  in  the  rushes  he  dreams  he  is  on  a  raft 

52%  the  boy's  dreams  are  in  conflict  with  what  others  expect  of  him 

6%  the  boy  prefers  the  night  when  sleep  and  dreams  occur 

Here,  students  must  be  aware  of  the  purpose  of  the  title,  to  further  enhance 
the  author's  meaning.  The  title,  "The  Dreamer,"  establishes  the  character  of 
the  boy  in  the  poem;  it  does  not  refer  to  the  content  of  the  poem.  From  their 
responses  to  this  question  and  to  "When  I  Read,"  it  appears  that  students 
have  difficulty  recognizing  the  function  of  titles  in  poetry. 
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A  Question  of  Answers 


These  poems  are  taken  from  Class  Dismissed,  a  collection  of  poems 
about  high  school  students.  Written  by  a  high  school  teacher,  these 
poems  were  inspired  by  the  students  in  his  classes.  Read  to  see  if  you 
are  reminded  of  any  of  your  classmates  or  maybe  even  yourself!  After 
reading,  please  answer  the  questions  that  follow. 


Dana  Moran 


Amanda  Butler 


I  was  in  such  a  hurry  to  get  out  of  school 

that  I  took  a  shortcut. 

I  dropped  out. 

Now  I  work  full-time  at  the  local  K  Mart 

Putting  out  stock,  handling  the  register, 

Dealing  with  the  crazy  customers 

Who  hassle  me  about  returns, 

Who  cut  the  line, 

Who  think  I  cheat  them  by  ringing  up 

the  wrong  prices. 
Taking  stock  of  what  I've  done 
I  see  that  the  shelves  of  my  life  are 

kinda  empty, 
Bare,  in  fact. 
When  will  it  register  that 
There  are  no  express  lines  to  happiness? 
I  am  such  a  fool. 
Check  it  out. 


Most  mothers  yell,  scream  or  hyperventilate. 

They  cajole,  wheedle  or  plead. 

They  bribe,  coerce  or  threaten. 

Not  my  mother. 

She  has  replaced  the  telephone 

With  the  refrigerator, 

Leaving  short,  cryptic  notes  on  the  door. 

Held  in  place  by  those  silly  little  magnets 

Which  look  like  overripe  fruits 

A  few  well-chosen  words  determine  my  daily  fate. 
"Be  home  at  four— sharp!" 
"Clean  up  your  room — today!" 
"Start  dinner— I'll  be  late." 
"You're  grounded — for  two  weeks." 
"Don't  you  know  what  a  curfew  means?" 
"I  just  got  the  phone  bill — are  you  crazy?" 

Smart  lady,  my  mother. 

There  is  never  a  chance  to  answer  back. 

"Leave  a  note,"  she  says 

As  she  flies  out  the  door  to  work. 

It's  not  the  same  thing. 

Refrigerator  conversations  leave  me  cold. 


From  CLASS  DISMISSED  II  by  Mel  Glenn. 

Text  copyright  ®  1986  by  Mel  Glenn. 

Reprinted  by  permission  of  Clarion  Books/Ticknor  &  Fields, 

a  Houghton  Mifflin  Company. 


1.  Which  line  of  the  poem  "Dana  Moran"  best  expressed  the  theme  of 
the  poem?  Please  explain  your  choice. 

2.  In  the  poem  "Amanda  Butler,"  why  doesn't  Amanda  "Leave  a  note," 
as  her  mother  suggests? 

3.  Explain  how  the  poet  uses  a  similar  technique  in  the  last  lines  of  both 
these  poems. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


Dana  Moran  /  Amanda  Butler 

Paired  poems  figure  in  the  next  exercise  also,  this  time  for  Grade  12  students. 
Poetry  remained  a  "success"  area  among  respondents  at  this  level.  Over  80 
percent  were  able  to  choose  a  line  expressing  the  theme  of  "Dana  Moran" 
and  justify  their  choice.  Only  6  percent  skipped  the  question  or  gave  ir- 
relevant answers.  "There  are  no  express  lines  to  happiness"  was  the  choice 
of  42  percent,  leaning  toward  the  impersonal.  Over  a  quarter  selected  the 
more  concrete  "The  shelves  of  my  life  are  kind  of  empty"  to  embody  the  same 
theme. 

Regarding  the  question  about  leaving  notes,  approximately  three-quarters 
inferred  Amanda's  attitude  with  no  hesitation.  "Amanda  doesn't  feel  that 
writing  a  note  means  anything,"  noted  one  student.  "The  intimacy  of  conver- 
sation is  missing  from  their  relationship,"  another  stated.  A  tendency  to  take 
sides  was  apparent,  however.  Student  comments  about  the  character  of  the 
mother  could  have  come  from  Amanda  herself.  "Her  mother  probably 
couldn't  be  bothered  to  listen  to  her  anyway"  was  one  example.  Another 
student  asserted  that  "Parents  never  seem  to  have  time  to  talk  to  us  kids  about 
what's  going  on." 

The  final  question  asked  students  to  assume  an  external  perspective  by 
analyzing  the  use  of  a  pun  in  the  last  line  of  each  poem.  Almost  a  quarter 
used  the  correct  term.  "The  poet  uses  puns:  speaking  about  a  cashier  he  uses 
'Check  it  out,'  and  when  speaking  about  'refrigerator  conversations'  says 
'leave  me  cold.'"  A  small  percentage  described  the  function  of  the  last  line 
accurately  but  labeled  it  "irony,"  "play  on  words,"  or  some  other  such  term. 
Another  third  were  able  to  discuss  the  last  lines  cogently  without  naming  the 
literary  device  in  question.  "The  first  poem  states  'check  it  out,'  meaning  to 
look  into  it,"  wrote  a  typical  student  in  this  category,  "not  'check  it  out'  like 
what  you  do  at  a  store." 

All  in  all,  two-thirds  of  the  respondents  handled  this  exercise  competently, 
relating  elements  to  total  text. 
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A  Question  of  Answers 


The  Wind  and  the  Sun 

Read  this  story  to  find  out  how  a  very  interesting  bet  was  won.  Then  answer 
the  question  that  follows. 


The  North-wind  and  the  Sun  once  had  an 
argument  as  to  which  had  the  more  power. 
They  were  squabbling  back  and  forth  with 
words,  words,  words  when  an  unlucky  man 
happened  to  walk  along  on  the  road 
beneath  them. 

"All  right,"  said  the  North-wind,  "here's  a 
chance  to  prove  who's  right.  I'll  bet  you  I 
can  take  that  man's  coat  off  more  quickly 
than  you  can." 

"That's  a  bet,"  said  the  Sun,  "you  try  first." 

The  North-wind  blew  an  icy  puff  at  the  man 
and  nearly  froze  him,  so  he  clutched  his 
coat  closer  around  him.  Then  the  North- 
wind  got  angry  and  puffed,  blew  and 
blasted  away  so  hard  he  even  brought  icy 


showers  down  on  the  miserable  man, 
whose  teeth  rattled  with  the  cold. 

Naturally  the  poor  fellow  hugged  his  thin 
little  coat  all  the  tighter  and  the  North- 
wind  had  to  give  up. 


"Now,  it's  my  turn,"  said  the  Sun. 
it  the  easy  way." 


'I'll  do 


He  came  out  from  behind  a  cloud  and 
started  to  shine  with  all  the  heat  he  could. 

After  a  short  while  the  man  was  so  warm 
he  had  to  take  off  his  coat  and  go  sit  in  the 
shade  of  some  trees.  So  the  Sun  won  the 
bet. 


1.  Write  your  own  moral  (or  lesson)  for  this  fable. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


Wind  and  Sun 


With  a  short  illustrated  fable  called  "The  Wind  and  the  Sun"  we  return  to  a 
Grade  4  exercise.  The  task  set  for  students  was  to  formulate  a  generalization 
from  the  text. 

When  a  general  proposition  was  stated  for  them,  in  the  form  of  a  multiple- 
choice  item  applied  to  the  same  text,  a  majority  of  respondents  were  able  to 
respond  correctly. 

What  would  be  the  BEST  "moral"  for  this  story? 

68%  It  is  better  to  use  your  brains  than  your  strength. 

6%  There's  always  somebody  whose  troubles  are  worse  than  your  own. 

1 7%  Sometimes  even  the  cleverest  people  are  caught  in  their  own  traps. 

10%  A  wise  man  learns  from  the  experience  of  others. 

When  the  task  was  presented  in  the  unstructured  format  of  the  open-ended 
question  shown  here,  42  percent  of  the  children  were  able  to  respond  with 
some  universal  truth.  Approximately  a  third  (31  percent)  suggested  that  the 
moral  to  be  drawn  was  not  to  challenge,  while  7  percent  suggested  a  moral 
more  or  less  along  the  lines  of  persuasion  being  better  than  force,  and  only  4 
percent  clearly  articulated  the  preferred  option  (brain  over  brawn)  in  the 
multiple  choice.  Students  appeared  to  perceive  the  story  in  terms  of  their  own 
life  experiences. 

On  the  other  hand,  some  students  may  have  been  unfamiliar  with  the  term 
"moral"  in  the  sense  intended  here  and  confused  it  with  what  they  understood 
by  "morality,"  which  in  turn  seemed  to  shade  into  what  could  be  termed 
"prudence."  Such  remarks  as  "Don't  make  a  bet  unless  you  know  you're 
going  to  win"  and  "Don't  brag  about  something  you're  not  sure  you  can  do" 
illustrate  this  point. 

The  majority  who  were  unable  to  answer  the  question  satisfactorily  went 
astray  for  a  variety  of  reasons.  In  some  cases  elementary  problems  of  reading 
comprehension  rather  than  inability  to  conceive  the  task  got  in  the  way. 
Morals  were  drawn  —  such  as  "It  doesn't  matter  who's  right  or  wrong,  it's 
how  you  play  the  game"  and  "What  it  is  trying  to  teach  you  is  that  the  sun  is 
stronger  than  anything  else"  —  but  they  did  not  reflect  the  meaning  of  the 
text. 
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A  Question  of  Answers 


Nevertheless,  abstraction  itself  was  a  problem  for  a  significant  fraction  of  the 
respondents.  Seizing  on  details  of  the  fable,  they  offered  what  might  be  called 
over-generalized  facts:  "You  can't  make  someone  take  off  his  coat  if  he's 
cold."  Literal-minded  students  looked  to  the  text  for  a  direct  answer  rather 
than  inferring  from  the  text.  They  reiterated  or  explained  what  they  saw  on 
the  surface:  "When  people  get  hot,  they  take  off  extra  clothing  to  try  and  get 
cooler." 

An  interesting  sidelight  is  that  a  number  of  fourth  graders  referred  to  the  wind 
as  a  cloud.  Arguably  the  illustration  that  accompanies  the  fable  is  ambiguous 
in  that  respect,  but  the  possibility  remains  that  any  illustration  may  obfuscate 
rather  than  clarify  matters  for  weak  readers  who  depend  on  pictures  to  "tell" 
them  the  story.  This  may  be  particularly  true  in  the  television  age,  when 
pictorial  figures  and  movements  are  presumed  —  indeed  designed  —  to 
dominate  over  words. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


Many  of  the  young  women  who  worked  in  the  Lowell  textile  mills  used 
the  power  of  the  press  to  call  attention  to  unsatisfactory  working  condi- 
tions. Below  is  the  first  half  of  a  factory  girl's  letter  to  the  editor  of  the 
Daily  Evening  Voice.  Read  it  and  answer  the  question  that  follows. 


Letter  from  a  Factory  Girl 


To  the  Editor  of  the  Daily  Evening  Voice: 

Lowell,  July  2,  1866 

A  victory  on  the  side  of  Labor  has  occurred  here,  which  I  fear  has  not  come 
under  the  notice  of  "Observer." 

I  will  try  and  give  you  the  account,  hoping  you  will  publish  it  for  the 
encouragement  of  others  who  labor  under  the  supervision  of  soulless  men. 
It  happened  in  the  "dressing  room"  of  the  Merrimack  Corporation.  [The 
"dressing  room"  is  where  yarn  is  treated  with  a  starch  paste  called  "size."] 

All  who  are  acquainted  with  the  mills  know  that  the  dressing  room  is  at 
the  best  an  uncomfortable  place  to  work.  In  the  room  mentioned  there 
are  fans  communicating  with  the  air  outside,  and  when  running  they  add 
much  to  the  comfort  of  the  dressers  [the  women  treating  the  yarn  with  size] . 
But  it  required  a  little  more  water  to  run  the  fans,  so  the  agent  determined 
they  should  be  stopped.  Not  satisfied  with  this,  he  also  ordered  that  the 
girls  should  lift  or  "put  up"  their  own  size.  This  would  enable  him  to 
discharge  from  two  to  four  men. 

But  the  girls,  determined  not  to  submit,  called  a  meeting  and  voted 
unanimously  not  to  start  their  frames  till  the  size  was  put  up  for  them  and 
the  fans  set  in  motion. 


1 .  Imagine  yourself  as  the  factory  girl  and  complete  the  letter  explaining 
how  the  crisis  in  the  dressing  room  of  the  Merrimack  Corporation 
was  resolved.  Please  base  your  "ending"  on  evidence  from  the 
unfinished  letter. 

Your  ending: 
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A  Question  of  Answers 


Letter  from  a  Factory  Girl 

A  portion  of  a  letter  from  a  19th-century  textile  worker  served  as  the  basis  for 
a  challenging  Grade  8  exercise.  To  deal  well  with  the  question  posed  here, 
students  had  to  handle  complicated  tasks  simultaneously.  (1)  They  had  to 
infer  the  author's  viewpoint,  stepping  back  a  bit  from  the  text  to  infer  also  her 
strategy  for  communicating  with  her  readers  of  that  time.  (2)  In  order  to 
complete  the  letter,  their  writing  had  to  be  stylistically  consistent  with  hers 
and  true  to  the  epistolary  genre.  (3)  To  make  sense  of  the  events  related,  they 
had  to  recognize  that  the  letter  states  that  the  factory  girls  won  a  victory. 

Although  two-thirds  of  the  students  completed  the  letter  in  a  stylistically 
consistent  manner,  only  half  recognized  that  the  girls  had  won  a  victory.  The 
most  successful  respondents  used  sophisticated  techniques  such  as  echoing 
phrases  from  the  opening  of  the  letter.  The  following  is  a  good  example. 

A  representative  from  that  meeting  was  sent  to  the  agent  to 
demand  their  comforts,  or  the  girls  would  not  start  their  frames. 
Since  this  would  hurt  the  factory,  the  agent  submitted.  Soon  the 
size  was  put  up  and  the  fans  set  in  motion,  showing  a  victory  for 
the  factory  girls. 

More  commonly,  students  grasped  some  events  but  not  the  language  and 
character  of  the  author  or  of  the  time.  The  following  example  falls  into  that 
class. 

...They  had  finally  accomplished  the  Impossible  Dream.  The  next 
day  the  boss  said,  "I  am  very  proud  of  you,  my  girls,  and  you  all 
deserve  a  raise  of  two  cents."  The  girls  were  very  happy,  two 
cents  could  buy  a  lot  more  than  what  they  had  ever  imagined.  All 
because  of  the  impossible  dream. 

Definitions  were  included  with  the  text  of  the  letter.  Some  students,  however, 
presuming  they  knew  what  "size,"  "frame,"  and  "dressing  room"  meant, 
ignored  both  the  definitions  and  the  context  that  necessitated  them.  "We  now 
have  the  fans  set  in  motion  and  they  are  putting  up  the  size  so  we  have  made 
the  frames"  was  a  typical  vocabulary  error.  In  view  of  such  results,  teachers 
might  stress  context  clues  and  elasticity  of  language  as  part  of  vocabulary 
training. 
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Approximately  10  percent  of  the  respondents  short-circuited  the  assignment 
by  offering  opinions  and  strategies  outside  the  letter  format.  Instead  of 
completing  a  letter  and  without  trying  to  speak  through  the  factory  girl,  they 
simply  gave  advice  on  the  situation  in  their  own  voices.  For  example:  "The 
Merrimack  Corporation  in  the  dressing  room  should  get  extra  help,  and  then 
everyone  would  be  satisfied."  It  is  interesting  to  note  that  in  a  converse 
assignment  —  the  Lake  Champlain  exercise  —  the  same  proportion  of 
students  err  in  the  opposite  direction.  When  asked  to  discuss  in  their  own 
words  what  a  newspaper  article  might  contain,  a  tenth  of  the  respondents 
attempted  to  write  the  rest  of  the  article. 
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A  Question  of  Answers 


Read  the  following  story  to  learn  howAntoine  Parmentier  convinced  the 
French  people  that  potatoes  were  fit  to  eat,  and  then  answer  question 
1. 


Potatoes 


According  to  an  old  story,  there  lived 
in  France  long  ago  a  scientist  named 
Antoine  Parmentier.  For  many  years 
he  had  tried  to  get  the  French  farmers 
to  plant  potatoes.  But  in  those  days 
few  people  had  even  eaten  potatoes. 
"Food  that  grows  under  the  ground 
would  only  be  fit  for  pigs,"  they  said. 

Then  there  came  a  time  in  France 
when  the  wheat  crops  failed  and  many 
people  were  hungry.  Once  again  Par- 
mentier suggested  his  idea  for  growing 
potatoes.  But  people  still  laughed  and 
made  fun  of  him.  Instead  of  becoming 
discouraged,  the  scientist  went  to  visit 
the  king.  He  explained  to  the  king  how 
potatoes  were  grown  and  eaten  in 
many  other  countries.  He  then  told 
the  king  of  a  plan  he  had,  and  the  king 
agreed  to  help  him  carry  it  out. 


Some  potatoes  were  secretly  planted 
in  a  field  near  Paris.  Day  and  night 
the  field  was  guarded  by  soldiers. 
People  called  it  the  "treasure  gar- 
den." When  the  potatoes  were  ripe, 
the  guards  were  taken  away.  That 
very  night  the  Parisians  sneaked  into 
the  field  and  dug  up  the  potatoes. 
Then  the  "thieves"  hurried  home  to 
cook  and  eat  the  strange  vegetable. 
Lo  and  behold,  everyone  liked  them! 

The  news  traveled  from  one  end  of 
France  to  the  other.  Before  many 
years  had  passed,  potatoes  could  be 
found  in  almost  every  French  home. 
Antoine  Parmentier  had  won  at  last. 


1 .  Explain  how  Antoine  Parmentier's  plan  reveals  his  understanding  of 
human  nature. 
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Potatoes 


An  exercise  titled  "Potatoes"  was  administered  to  Grade  8  students  and 
Grade  12  students  in  both  open-ended  form  and  multiple-choice  form. 
Intriguing  results  emerged. 

In  advance  "Potatoes"  promised  to  be  a  relatively  difficult  assignment,  espe- 
cially for  Grade  8.  Students  were  challenged  to  comprehend  a  main  theme 
and  to  articulate  a  generalization,  and  coming  to  grips  with  the  abstraction 
"human  nature"  was  essential  to  the  task.  When  choosing  among  multiple- 
choice  alternatives,  younger  respondents  did  indeed  run  into  trouble,  as  the 
following  percentages  indicate: 

1 9%  People  want  what  they  cannot  have. 

2%  The  people  of  Paris  were  thieves. 

52%  Potatoes  are  fine  for  people  to  eat. 

27%  Antoine  Parmentier  was  very  clever. 

The  performance  of  older  students  as  a  group,  although  better,  was  likewise 
shaky  when  they  chose  among  statements  of  theme: 

38%  People  want  what  they  cannot  have. 

3%  The  people  of  Paris  were  thieves. 

30%  Potatoes  are  fine  for  people  to  eat. 

30%  Antoine  Parmentier  was  very  clever. 

Surprisingly,  when  it  came  to  approaching  the  task  through  the  open-ended 
question,  half  the  Grade  8  respondents  did  well.  "People  covet  what  others 
value  or  guard"  and  "People  don't  want  to  try  new  things"  were  typical 
responses.  In  the  open-ended  format,  nearly  three-quarters  of  the  Grade  12 
respondents  handled  human  nature  with  aplomb.  Concise,  sophisticated 
answers  were  common.  The  phrase  "forbidden  fruit"  turned  up  several  times. 
Some  students  who  perhaps  fell  short  of  the  appropriate  level  of  generality 
compensated  with  eloquence.  The  following  is  an  example: 

He  saw  that  even  in  times  of  need  people  would  not  concede  to 
what  they  considered  to  be  animal-like  behavior  in  order  to  sur- 
vive. He  understood  that  people  have  a  need  to  feel  important  and 
that  they  try  to  preserve  their  dignity.  By  planting  the  potatoes  in  a 
small  field  guarded  by  the  King's  soldiers  he  created  an  aura  of 
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A  Question  of  Answers 


"royal"  importance  about  them.  Even  in  times  of  famine  people 
would  much  rather  eat  exotic  foods  swiped  from  the  King's 
"treasure  garden"  than  food  grown  in  dirt  that  is  fit  only  for  pigs. 

The  results  took  still  another  unexpected  turn.  On  the  whole,  those  respon- 
dents who  did  not  do  exceptionally  well  did  exceptionally  poorly.  The  middle 
group  disappeared. 

From  the  "other  half  of  the  Grade  8  group  came  a  high  number  of  inap- 
propriate or  irrelevant  answers,  evidently  stemming  from  a  misinterpretation 
of  the  expression  "human  nature."  Not  recognizing  it  as  a  unit,  a  number  of 
students  construed  it  to  mean:  "human  =  Parmentier,"  "nature  =  potatoes." 
This  produced  statements  on  the  order  of:  "That  the  potatoes  were  grown 
under  the  ground,  that's  human  nature  —  from  all  the  roots  and  everything" 
and  "It  reveals  his  understanding  of  human  nature  because  he  understands 
that  because  something  is  grown  underground  doesn't  mean  it's  pig  food." 
The  king's  role  was  frequently  over-estimated,  as  in  "He  knew  that  people 
wouldn't  believe  him,  so  he  asked  the  king  for  help,  and  of  course  people  had 
respect  for  the  king  so  they  tried  them."  A  few  students  were  beguiled  by  the 
expression  "fit  for  pigs"  and  depended  on  it  for  explanatory  power,  as  in  the 
following  comment: 

Antoine's  plan  reveals  human  nature  in  a  way  that  is  saying  you 
never  know  until  you  try  and  that  pigs  are  people  too.  Doesn't  mat- 
ter what  it  is  as  long  as  you  like  it. 


A  disappointing  20  percent  of  the  Grade  8  respondents  looked  for  the  answer 
within  the  text.  They  "explained"  the  story  by  reiterating  it,  along  the  lines 
of  "He  had  planted  them  in  a  field  near  Paris  and  had  soldiers  watch  them." 
Nearly  as  many  Grade  12  respondents  (17  percent)  reiterated  the  story  too, 
but  fewer  resorted  to  other  kinds  of  irrelevant  or  inappropriate  answers. 

When  compared  to  the  multiple-choice  item,  the  open-ended  question  ap- 
peared to  be  the  more  difficult.  It  calls  for  not  only  an  understanding  of  the 
main  theme,  but  an  articulated  generalization.  However,  the  results  indicate 
that  students  performed  more  successfully  in  the  less  structured  task  than  they 
did  when  presented  with  a  set  of  pre-selected  alternatives. 

The  open-ended  question,  focussing  as  it  did  on  "human  nature,"  appeared 
to  indicate  the  need  for  abstraction.  When  asked  only  to  "explain  a  theme," 
students  reverted  back  to  seeking  the  answer  within  the  text.  Convincing 
people  that  potatoes  are  fine  for  people  to  eat  was  Parmentier's  stated 
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purpose  in  the  story;  it  not  the  purpose  of  the  story.  Students  were  asked  to 
use  inferences  to  draw  conclusions  (evaluating  and  extending  meaning). 
Instead,  they  answered  on  the  level  of  textual  analysis.  From  the  open-ended 
question,  we  have  evidence  that  students  are  capable  of  abstraction.  What 
may  be  needed  is  direction  in  recognizing  when  abstraction  is  required. 
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A  Question  of  Answers 


The  Lake  Champlain  Monster 


A  newspaper  article  is  entitled  "Lake  Champlain's  Monster  —  Fact  or 
Fiction?"  The  first  two  sentences  of  the  article  are: 

"Believers  say  the  warm  waters  of  summer  bring  the  monster  to  the 
surface.  Others  say  that  the  monster  is  just  the  creation  of  jokesters." 

Describe  the  kind  of  information  you  expect  the  rest  of  the  article  to 
contain. 


The  open-ended  exercise  called  "The  Lake  Champlain  Monster"  asks  stu- 
dents to  associate  genre  with  its  typical  characteristics.  At  fourth  grade,  the 
question  does  not  ask  children  to  predict  what  happens  next.  However,  this 
is  how  most  children  interpreted  it.  Almost  half  completed  the  article  with 
their  own  story,  while  one-quarter  were  able  to  suggest  that  two  sides  would 
be  presented.  Typical  of  each  category  are: 

Some  say  the  monster  is  friendly  and  others  don't.  They  also  say 
it  is  233  years  old.  It  has  yellow  eyes  and  a  green  body  with  pokes 
on  its  back.  It  comes  out  every  three  hours.  Some  people  don't 
believe  in  it  but  just  in  case  it  is  real,  they're  staying  away  from  the 
water. 


The  article  might  describe  the  monster.  The  article  might  name 
some  people  who  have  seen  the  monster.  It  may  explain  how  the 
monster  comes  to  the  surface.  The  article  may  contain  facts 
about  why  there  might  be  a  monster  and  why  there  might  not  be  a 
monster. 
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A  newspaper  article  is  entitled  "Lake  Champlain's  Monster  —  Fact  or 
Fiction?"  The  first  two  sentences  of  the  article  are: 

"Believers  say  the  warm  waters  of  summer  bring  the  demon  to  the  sur- 
face. Skeptics  suggest  the  dark  dragon  responds  better  to  the  ringing 
of  cash  registers." 

Describe  in  detail  the  kind  of  information  you  expect  the  rest  of  the 
article  to  contain. 


The  other  version  of  the  question  challenged  Grade  8  students  and  Grade  12 
students  to  make  relatively  large  inferences  from  a  fragment  of  text.  Inter- 
pretation of  figurative  language,  familiarity  with  newspaper  feature  writing 
as  a  genre,  and  expectations  about  presentation  of  evidence  all  had  a  bearing 
on  the  assignment. 

A  number  of  Grade  8  respondents  seemed  unable  to  think  metaphorically, 
and  they  took  the  ringing  of  cash  registers  literally.  "The  creature  was 
sensitive  to  the  ringing,"  one  student  concluded.  "We  will  be  taking  the  bells 
out  of  cash  registers  for  the  summer  and  doing  something  to  stop  the  water 
from  being  warm,"  another  wrote.  Other  Grade  8  respondents  recognized 
the  figurative  language  but  misjudged  the  writer's  tone  and  constructed 
fanciful  metaphors  of  their  own.  "It  will  probably  also  contain  some  facts  on 
how  each  consumer  probably  has  one  of  these  dark  dragons  in  them  that  likes 
to  spend  money"  fell  into  this  group.  A  related  example:  "I  would  expect 
information  concerning  the  economy  and  possible  economic  problems  in  the 
Lake  Champlain  territory.  The  monster  is  a  symbol  of  misfortune  and  a 
struggling  flow  of  money." 

A  significant  number  of  students  (22  percent  at  Grade  8  and  15  percent  at 
Grade  12)  completed  the  article  or  completed  a  fiction  piece  and  in  so  doing 
exposed  a  certain  vagueness  as  to  the  components  of  an  "article."  Several 
students  interpreted  the  title  as  a  question  for  them  to  answer,  circling 
"Fiction." 

Although  about  half  experienced  difficulty  with  the  details  of  the  assignment, 
Grade  12  respondents  had  a  better  understanding  in  general  of  what  was 
expected  of  them.  The  majority  more  or  less  recognized  metaphor,  genre, 
and  the  call  for  critical  inference. 
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About  a  quarter  of  the  Grade  12  respondents  saw  the  text  as  an  intriguing 
lead  into  a  serious  story  in  which  opposing  evidence  would  be  cited.  They 
answered  the  question  in  the  following  spirit: 

I  expect  the  rest  of  the  article  to  contain  certain  points  of  facts  that 
disprove  the  existence  of  this  dragon  and  other  points  that  sup- 
port the  theory  of  the  existence  of  the  dragon.  There  will  probably 
be  people  interviewed  that  believe  and  disbelieve  in  the  dragon, 
and  it  may  include  a  reason  why  people  may  believe  this  dragon 
lives. 

The  rest  of  the  article  would  contain  eyewitness  reports  of  sight- 
ings of  the  monster.  The  reports  would  give  descriptions  of  the 
monster,  the  time  of  day  the  sighting  occurred,  the  date,  the 
weather,  and  any  unusual  movements  the  monster  made.  The  ar- 
ticle would  also  include  scientific  facts  that  the  monster  does  not 
exist. 


A  different  quarter  of  Grade  12  respondents  recognized  the  genre  and 
understood  the  assignment  as  well  as  the  first  quarter  referred  to  above. 
Because  of  cynicism  about  journalism,  however,  they  tended  to  view  the 
presentation  of  "evidence"  as  just  a  formality  in  such  articles.  Typical  com- 
ments from  this  group: 

The  story  probably  doesn't  contain  anything  about  a  real  monster. 
They  just  put  it  that  way. 

Choppy  sentences  full  of  nonsense  facts  conjured  up  by  a  strug- 
gling journalist  trying  to  make  a  name  for  himself. 

I'd  expect  an  explanation  of  the  two  positions:  believers  and  skep- 
tics. There  would  be  some  eyewitness  accounts  from  believers 
and  some  really  cynical  remarks  from  the  skeptics  about  how  it's 
all  a  publicity  stunt  to  get  people  to  come  and  spend  money  at  the 
lake.  Judging  from  the  title,  the  article  would  probably  end  with 
some  sort  of  corny  restatement  of  the  first  two  sentences. 
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Mary  McLeod  Bethune 


A  book  begins  "Mary  McLeod  Bethune  was  born  in  Mayesville,  South 
Carolina  on  July  10, 1875."  Describe  the  kind  of  information  you  expect 
the  book  to  contain. 





A  sentence  about  the  birth  date  of  Mary  McLeod  Bethune  —  the  opening  of 
a  simply  written  biography  —  was  presented  to  Grade  4  students  to  test  their 
evaluation  of  type  and  structure. 

Although  most  students  have  probably  not  been  exposed  to  this  kind  of 
activity,  75  percent  of  the  respondents  did  anticipate  a  biography  from  the 
single  sentence.  "It  would  be  a  biography,"  wrote  one  confident  student,  "a 
story  to  tell  about  a  person's  life." 

Of  the  students  who  succeeded  with  this  exercise,  10  percent  showed  an 
understanding  of  the  rationale  for  writing  biography.  "It  should  contain 
information  about  her  life  and  why  she  was  famous.  It  should  also  have  when 
she  died  and  why,"  answered  a  respondent  in  this  group.  A  few  respondents 
took  pains  to  mark  biography  off  from  the  other  genres,  as  in  the  following 
example: 

It  would  probably  contain  information  about  her  life  and  some 
great  things  she  did,  but  that's  only  if  it  was  a  biography.  It  could 
instead  contain  some  sort  of  crazy  adventure,  experience,  or  just 
about  anything  if  it  was  a  fiction  book. 

Predictably,  some  students  completed  the  "story"  instead  of  standing  back 
from  the  text.  "Mary  McLeod  Bethune  is  a  author,"  according  to  one  of  those 
students.  "She  was  a  happy  woman.  She  had  two  kids  and  a  wonderful 
husband." 

A  corresponding  multiple-choice  question  that  narrowed  the  respondents' 
task  down  to  choosing  among  definitions  produced  commensurate  over-all 
results. 

A  story  about  a  person's  life  is  called 

1 7%  fiction 

2%  a  poem 

79%  a  biography 

1  %  a  fairy  tale 
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A  Question  of  Answers 


Three  Views  of  the  Lowell  Mills 


Questions  1-3:  Read  all  three  of  the  following  viewpoints  before  answering  the  questions. 

Factory  owners  claimed  that  their  operatives  (as  factory  workers  were  called)  enjoyed  ideal 
working  and  living  conditions.  Nevertheless,  in  1836  many  operatives  formed  unions  and  went 
out  on  strike.  Their  actions  were  unsuccessful.  When  the  workers  ended  their  strike,  they  found 
out  union  leaders  had  been  fired.  Soon  after  the  strike  of  1836,  a  visitor  wrote  about  the  factories 
of  Lowell.  Read  the  following  passage  to  learn  about  some  of  the  conditions  that  lead  workers 
to  form  labor  unions. 


View  1:  An  American  Journalist 


The  operatives  work  thirteen  hours  a  day 
in  the  summer  time,  and  from  daylight  to 
dark  in  the  winter.  At  half  past  four  in  the 
morning  the  factory  bell  rings,  and  at  five 
the  girls  must  be  in  the  mills.  A  clerk, 
placed  as  a  watch,  observes  those  who  are 
a  few  minutes  behind  the  time,  and  ... 
[steps]  are  taken  to  ...  [encourage] 
punctuality.... 

As  seven  the  girls  are  allowed  thirty 
minutes  for  breakfast,  and  at  noon  thirty 
minutes  more  for  dinner,  except  during  the 
first  quarter  of  the  year,  when  the  time  is 
extended  to  forty-five  minutes....  At  seven 
o'clock  in  the  evening  the  factory  bell 
sounds  the  close  of  the  day's  work. 

So ...  exhausted  and  worn  out ...  are  a  num- 
ber of  girls  that  they  go  to  bed  soon  after 
their  evening  meal.... 


Source:  The  Harbinger,  Nov.  14,  1836,  in  Bailey  and  Kennedy 
(ads.),  The  American  Spirit,  Vol  1.  pp.  278-280. 


Now  let  us  examine  the  nature  of  the  labor 
itself,  and  the  conditions  under  which  it  is 
performed.  Enter  with  us  into  the  large 
rooms,  when  the  looms  are  at  work....  [In 
one  factory]  the  din  and  clatter  of  ...  five 
hundred  looms,  under  full  operation, 
struck  us  on  first  entering  as  something 
frightful  and  infernal....  After  a  while  we 
became  somewhat  inured  [accustomed]  to 
it,  and  by  speaking  quite  close  to  the  ear  of 
an  operative  and  quite  loud,  we  could  hold 
a  conversation  and  make  the  inquires  we 
wished. 

The  girls  attend  upon  [take  care  of]  an 
average  of  three  looms;  many  attend  four, 
but  this  requires  a  very  active  person,  and 
the  most  unremitting  [constant]  care. 

...  The  atmosphere  of  such  a  room  cannot 
of  course  be  pure;  on  the  contrary,  it  is 
charged  [filled]  with  cotton  filaments  and 
dust,  which,  we  are  told,  are  very  injurious 
[harmful]  to  the  lungs. 
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Not  all  visitors  to  the  Lowell  mills  were  critical  of  the  working  conditions.  Read  the  following 
accounts  by  Davy  Crockett  and  Captain  Hall,  an  Englishman,  for  a  different  perspective,  and 
then  answer  the  questions  that  follow. 


View  2:  Davy  Crockett 


View  3:  Captain  Hall 


We  stopped  at  a  large  stone  house  at  the 
head  of  the  falls  of  the  Merrimack  River, 
and  having  taken  a  little  refreshment,  went 
down  among  the  factories.  The  dinner- 
bells  were  ringing  and  the  folks  pouring  out 
of  the  houses  like  bees  out  of  gum.  I  looked 
at  them  as  they  passed,  all  well-dressed, 
lively  and  genteel  in  their  appearance,  in- 
deed the  girls  looked  as  if  they  were  coming 
from  a  quilting  frolic. ...  I  went  in  among 
the  young  girls,  and  talked  with  many  of 
them.  Not  one  expressed  herself  as  tired 
of  her  employment,  or  oppressed  with 
work;  all  talked  well,  and  looked  healthy. 
Some  of  them  were  very  handsome;  and  I 
could  not  help  observing  that  they  kept  the 
prettiest  inside,  and  put  the  homely  ones 
on  the  outside  rows. 


from  Crockett's  Account  of  a  Tour  to  the  North  and  Down  East, 
1634. 


The  whole  discipline,  ventilation,  and 
other  arrangements  appeared  to  be  excel- 
lent, of  which  the  best  proof  was  the  heal- 
thy and  cheerful  look  of  the  girls,  all  of 
whom,  by  the  way,  were  trimmed  out  with 
much  neatness  and  simplicity,  and  wore 
high  tortoise-shell  combs  at  the  back  of 

their  heads On  the  13th  of  October,  at 

6  o'clock  in  the  morning,  I  was  awakened 
by  the  bells  which  tolled  the  people  to  their 
work,  and  on  looking  from  the  window, 
saw  the  whole  space  between  the  factories 
and  the  village  speckled  over  with  girls, 
nicely  dressed,  and  glittering  with  bright 
shawls  and  showy-colored  gowns  and  gay 
bonnets,  all  streaming  along  to  their  busi- 
ness, with  an  air  of  lightness,  and  an  elas- 
ticity of  step,  implying  an  obvious  desire  to 
get  to  their  work. 


from  Captain  Hall's  Travels  in  North  American  in  the  Year  1827 
and  1828. 


Questions  1-3:  Some  of  the  differences  in  these  three  views  of  the 
conditions  in  the  Lowell  Mills  may  be  explained  by  the  different  back- 
grounds of  the  three  observers:  View  1  by  an  American  journalist,  View 
2  by  an  American  frontiersman,  and  View  3  by  an  English  military  man. 
Evaluate  to  what  extent  each  view  may  be  biased  by  the  observer's 
background.  Please  explain  your  reasons  in  each  case. 

1 .  View  1 :  An  American  Journalist 

2.  View  2:  Davy  Crockett 

3.  View  3:  Captain  Hall 


32 


A  Question  of  Answers 


Three  Views  of  the  Lowell  Mills 


In  an  exercise  for  Grade  12,  three  very  different  accounts  of  the  Lowell  mills 
by  men  who  visited  there  were  set  side  by  side  to  test  perception  of  bias  or 
viewpoint.  The  pattern  of  response  was  unusually  clear-cut. 

By  Grade  12  a  majority  of  students  appear  able  to  apply  life  experience  to 
reading  and  thinking,  and  some  offer  astute  observations  of  people's  roles 
and  motives.  Two-thirds  of  those  who  responded  to  this  question  readily 
found  and  accounted  for  bias  in  the  texts.  Among  the  impressive  answers 
were  the  following: 

Viewl:  An  American  Journalist 

The  journalist  probably  exaggerated  the  conditions  of  the  Lowell 
mills,  as  a  journalist  would  use  hyperbole  to  make  his  story  inter- 
esting. The  use  of  hyperbole,  though,  does  make  one  envision  a 
group  of  girls  who  work  incredibly  hard  at  this  job. 

View  2:  Davy  Crockett 

To  the  best  of  my  knowledge,  Davy  Crockett  was  not  an  expert  in 
the  field  of  mill  evaluation.  He  was  an  explorer  and  frontiersman. 
In  my  own  opinion,  I  think  Davy  Crockett  was  traveling  with  no 
female  companion.  I  think  he  was  more  concerned  with  the 
women  themselves  than  with  the  conditions  surrounding  their 
work  and  the  mill. 

View  3:  Captain  Hall 

The  Captain  is  a  military  man  and  thus  is  used  to  people  being 
slightly  unhappy  with  their  lot  and  also  with  them  being  pressured 
into  a  regimen  of  work  and  discipline.  Thus  maybe  he  doesn't  feel 
that  this  is  so  terrible  for  civilians. 

Unfortunately,  the  students  who  have  not  developed  the  ability  to  read 
critically  are  separated  farther  than  ever  from  their  classmates  by  Grade  12. 
Though  their  vocabulary  has  grown  and  their  sentence  structure  has  im- 
proved, they  are  captives  of  the  Grade  4  impulse  to  look  for  an  answer  in  the 
text. 
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Thus  15  percent  of  the  respondents  to  the  Lowell  mill  question  did  not  discuss 
bias.  They  simply  recapitulated  the  passages.  "The  journalist  found  many 
things  to  be  harmful  and  dangerous"  ran  one  answer.  "Captain  Hall  describes 
the  women  as  cheerful  and  healthy"  went  a  similar  response.  That  analysis 
might  involve  more  than  literal  content  has  simply  not  sunk  in. 


Obviously  the  exercises  that  have  been  shown  here  are  neither  exhaustive 
nor  perfect.  The  purpose  of  considering  them  in  some  detail  is  to  invite 
teachers  to  think  concretely  about  analytical  and  evaluative  skills  that  need 
to  be  taught. 

An  active  reader  is  above  all  a  critical  reader.  He  or  she  is  capable  of 
approaching  the  same  text  with  different  ends  in  mind  and  switching  strategies 
smoothly  when  moving  from  one  type  of  material  to  another.  Even  fairly 
young  students  who  are  still  learning  to  "decode"  the  literal  information 
contained  in  a  text  can  begin  to  step  back  and  assess  the  reasonableness  of 
what  is  presented.  Extensive  practice  in  comparing  and  contrasting  texts  is 
desirable.  In  the  upper  grades,  especially  with  texts  that  marshal  evidence 
and  assert  conclusions,  nothing  less  than  reading  judgment  is  the  goal.  Like 
a  juror  in  the  courtroom,  a  critical  reader  weighs  the  preponderance  of 
evidence.  Does  it  support  an  author's  claim?  When  other  authors  put  forth 
counter-claims,  which  makes  the  best  case? 

To  teach  critical  reading  skills  is  an  ambitious  goal,  the  more  so  because  they 
must  be  joined  to,  not  ^uDStituted  for,  the  narrower  skills  traditionally  as- 
sociated with  reading  comprehension.  (As  students  go  out  into  the  world, 
they  will  probably  never  think  that  income-tax  forms,  say,  are  altogether 
"reasonable,"  but  they  must  be  able  to  follow  directions.)  The  potential 
rewards  of  teaching  critical  reading  skills,  however,  are  commensurate  with 
the  effort.  It  means  involving  students  in  forms  of  thinking  that  they  will  use 
again  and  again  in  their  lives. 

The  purpose  of  this  report  is  to  draw  attention  to  the  particular  merit  of 
open-ended  questions  in  constructing  lessons.  Appropriate  questioning  by 
the  teacher  is  a  path  to  appropriate  questioning  by  students. 


34 


Summary  and  Conclusions 


Summary  and 
Conclusions 


C 


lear  trends  emerge  from  the  results  that  have  both  developmental  and 
instructional  implications. 


Students  performed  reasonably  well  when  asked  to  interpret  poetry;  however, 
they  had  great  difficulty  with  its  formal  and  figurative  aspects.  Only  half  of 
the  fourth  graders  recognized  the  playful  use  of  language  in  the  poem  When 
I  Read.  Some  appeared  to  considered  the  title  as  the  first  line  of  the  poem; 
many  took  the  metaphor  of  racer  in  its  most  literal  sense.  While  older 
students  were  more  successful,  especially  when  asked  to  interpret  poetry,  they 
also  experienced  difficulty  in  recognizing  the  relationship  between  its  formal 
elements  and  meaning. 

To  step  back  from  the  text,  to  view  it  as  a  genre  in  itself  —  quite  distinct  from 
its  meaning  —  is  not  a  common  procedure  among  students  at  any  grade  level. 
Many  seem  to  be  unaware  of  formal  or  stylistic  devices  that  authors  use  to 
enhance  meaning.  Instead,  like  the  subject  of  When  I  Read,  they  appear  to 
"plunge  right  in." 

This  tendency  was  evident  in  non-fiction  as  well.  In  response  to  the  question 
on  The  Lake  Champlain  Monster,  a  moderate  proportion  of  students  at  both 
eighth  and  twelfth  grade  levels  found  difficulty  with  the  figurative  language 
involved,  misinterpreting  the  metaphor  used. 

In  addition  to  the  difficulty  they  showed  with  figurative  language  and  devices, 
students  did  not  appear  to  recognize  more  generalized  metaphors.  For  ex- 
ample, less  than  half  of  the  younger  children  were  able  to  give  a  reasonable 
moral  in  response  to  the  fable  The  Wind  and  the  Sun.  Many  read  the  text 
literally,  failing  to  understand  the  fantasy  involved  —  surprising  in  view  of  the 
role  of  fantasy  in  children's  literature. 

Although  the  ability  to  abstract  may  be,  in  large  part,  developmental,  difficul- 
ties persisted  throughout  the  grades.  When  asked  to  generalize  the  theme  of 
Potatoes,  for  example,  only  half  the  eighth  grade  students  and  three-quarters 
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of  the  twelfth  graders  gave  a  correct  response.  A  sizeable  proportion  at  both 
grade  levels  answered  literally,  merely  reiterating  the  passage.  Again,  they 
did  not  recognize  the  need  to  infer  from  the  text  rather  than  try  to  find  the 
answer  in  it. 

Older  students  were  most  successful  when  they  were  asked  to  infer  character. 
Eighth  grade  students  clearly  identified  with  the  boys  in  The  Cave  and  The 
Dreamer,  while  twelfth  graders  did  likewise  with  Dana  Moran  and  Amanda 
Butler. 

However,  their  understanding  was  not  confined  to  their  contemporaries. 
Although  half  did  not  read  the  question  carefully  enough  to  give  a  completely 
satisfactory  response,  two-thirds  of  the  eighth  grade  students  were  able  to 
identify  enough  with  the  character  in  Letter  from  a  Factory  Girl  to  write  a 
letter  in  an  acceptable  style  and  manner.  Responses  to  a  parallel  question  at 
the  twelfth  grade,  Three  Views  of  the  Lowell  Mills,  showed  an  impressive 
variety  of  valid  answers.  Many  students  appear  to  have  a  clear  picture  of 
people's  roles  and  are  able  to  apply  life  experience  to  reading  and  thinking. 


The  underlying  theme  of  the  questions  presented  here  is  their  demand  for 
an  active  engagement  with  the  text:  the  use  of  structural  cues,  the 
application  of  prior  knowledge,  and  the  continual  interplay  of  conjecture  and 
confirmation  to  get  at  the  central  meaning.  Although  some  of  the  abilities 
measured,  such  as  abstraction,  may  be  developmental  in  nature,  many  are 
developed  as  a  result  of  exposure.  That  students  perform  so  well  when  asked 
to  interpret  the  thoughts  and  actions  of  characters  argues  that  interest  and 
familiarity  play  a  large  role  in  their  performance.  Conversely,  their  difficul- 
ties with  the  more  external  aspects  of  reading  —  recognizing  metaphors, 
identifying  the  effect  of  the  structural  aspects  of  text,  generally  looking  at  the 
text  qua  text  —  suggests  a  lack  of  familiarity  with  this  way  of  reading. 

Confirmation  of  this  comes  from  a  committee  member  who  made  the  follow- 
ing comment  on  the  results  of  the  open-ended  assessment: 

I  feel  that  the  results  of  the  open-ended  questions  indicate  that  the 
skills  tested  have  not  been  presented  in  this  manner  and  style  to 
students.  For  example,  completing  a  letter  or  predicting  what  infor- 
mation will  be  included  in  subsequent  paragraphs  is  "foreign"  to 
most  students  at  all  levels  tested. 

Linda  McMenimen 

Teacher 

Greater  Lowell  Voc  Tech 
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Another  member  had  specific  suggestions  for  instruction.  Among  them  were : 

The  use  of  non-fiction  materials  -  teaching  through  the  use  of 
real  informational  articles  and  texts,  science,  social  studies  and 
real  life  forms  and  print  media. 

Teaching  of  study  strategies  -  plans  and  tools  for  learning,  prior 
knowledge,   purpose,  self-questioning,  student  active  processes. 

Writing  -   comprehension  skills  such  as  summarizing,  note- 
taking,  outlining,  etc.  which  forces  students  to  examine  and  or- 
ganize their  written  thoughts  and  put  the  author's  ideas  into  their 
own  language. 

Higher  level  thinking  skills  —  using  more  difficult  materials,  in- 
ferences, analyzing,  comparing,  evaluating  various  writers  and 
schools  of  thought,  but  through  teacher  modeling,  not  as  inde- 
pendent  practice. 

Edward  Sacco 

Director  of  Reading 

Reading  Public  Schools 


Another  member  made  the  following  comment  about  the  open-ended  ques- 
tion format: 

The  use  of  the  open-ended  question  format  in  the  assessment  test- 
ing is  a  positive  step  forward  toward  our  goal  of  graduating  com- 
petent young  adults  who  can  problem  solve  and  who  possess 
strong  communication  skills.  As  an  elementary  principal,  I  feel  the 
use  of  open-ended  questions  in  conjunction  with  other  good  test- 
ing techniques  should  begin  at  the  primary  level  and  develop  and 
grow  in  subsequent  grades. 

Joan  Sheehan 

Principal 

West  Springfield 
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The  open-ended  questions  discussed  in  this  booklet  are  classified  on  the  next 
page  according  to  the  Massachusetts  Framework  for  Reading  Comprehen- 
sion. This  framework  is  used  by  the  assessment  program  to  help  teachers 
diagnose  students'  strengths  and  weaknesses  in  reading  comprehension.  EX- 
TERNAL refers  to  the  formal  aspects  of  a  text,  in  contrast  to  its  meaning. 
Skills  classified  under  External  Analysis  and  Evaluation  include  recognizing 
the  literary  conventions  associated  with  different  types  of  reading  material 
and  being  able  to  adjust  reading  accordingly.  INTERNAL  refers  to  those 
skills  that  readers  use  to  extract  and  extend  their  understanding  of  the 
content.  They  are  similar  to  the  traditional  skills  associated  with  reading 
comprehension.  A  complete  discussion  of  the  framework  is  contained  in 
Reading  and  Thinking:  A  New  Framework  for  Comprehension. 

Following  the  framework  are  the  results  which  form  the  basis  for  discussion 
in  this  booklet.  They  are  presented  for  the  convenience  of  teachers  who  wish 
to  compare  their  students'  performance  with  those  of  students  statewide. 
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Analysis 

Evaluation 

Types  of  information: 

Grade  4:       Teeth 

E 

Relationship  of  elements: 

X 
T 

mi 

Grade  4:       Mary  McLeod  Bethune 

E 
R 
N 
A 
L 

Author's  purpose,  point 
of  view,  tone: 

Grade  8:       Letter  from  a  Factory 
Girl 

Use  of  evidence  to  draw  inferences: 

Grade  4:          Teeth 

Grade  4/8/12:  The  Lake  Champlain 
Monster 

Relationship  of  elements 

Grade  8:       The  Cave/The  Dreamer. 
Question  2 

Grade  12:     Dana  Moran/Amanda 
Butler.  Question  3 

Main  Ideas/Issues 

Grade  4:       The  Wind  and  the  Sun 

1 

Grade  8/12:  Potatoes 

N 
T 
E 

Grade  12:     Dana  Moran/Amanda 
Butler.  Question  1 

R 
N 
A 

L 

Reliability  of  sources: 

Grade  12:        Three  Views  of  the 
Lowell  Mills 

Use  of  evidence  to  draw  inferences: 

Grade  8:          When  I  Read.  Question  2 

Grade  8:          The  Cave/The  Dreamer. 
Question  1 

Grade  1 2:        Three  Views  of  the 
Lowell  Mills 

Grade  12:        Amanda  Butler/Dana 
Moran.  Question  2 
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The  Wind  and  The  Sun 


Reporting  Category:     Internal  Analysis 
Grade  Level:  Four 


Response  Summary:  Frequency  (%) 

a.  Appropriate  generalization:  42 

b.  Inappropriate  generalization  20 

c.  Answers  specifically  to  details  14 

d.  Reiterates  story  13 

e.  Other  inappropriate/  blank  1 1 

Examples: 

a.  Don't  make  a  bet  unless  you  know  you're  going  to  win. 
It's  better  to  have  a  brain  than  to  be  strong. 
Kindness  is  better  than  force. 

b.  If  you're  warm  and  friendly,  people  will  open  up  to  you. 

c.  You  can't  make  someone  take  off  his  coat  if  he's  cold. 

d.  The  sun  won  because  he  tried  to  warm  the  man  with  all  his  heart  and  the  wind  tried  to 
freeze  him. 
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The  Lake  Champlain  Monster 


Reporting  Category:        External  Evaluation 
Grade  Level:                    Four 

Response  Summary: 

Frequency  (%) 

a.  Detailed  evidence  on  both  sides 

6 

b  .  Vague  evidence  that  two  sides 
will  be  presented 

17 

c.  Evidence  of  one  side  presented 

25 

d.  Completion  of  article 

43 

e.  Irrelevant 

5 

f.    Blank 

5 

Examples: 

a.  The  article  might  describe  the  monster.  The  article  might  name  some  people  who 
have  seen  the  monster.  It  may  explain  how  the  monster  comes  to  the  surface.  The  ar- 
ticle may  contain  facts  about  why  there  might  be  a  monster  and  why  there  might  not 
be  a  monster. 

b.  The  writer  probably  tells  a^cul  both  sides  of  the  story.  First  he  might  say  what  the 
people  who  believe  say  and  then  what  the  people  who  don't  believe  say. 

c.  What  color  is  he.  How  big  is  he.  What  does  he  eat.  What  he  looks  like. 

d.  Some  say  the  monster  is  friendly  and  others  don't.  They  also  say  it  is  233  years  old.  It 
has  yellow  eyes  and  a  green  body  with  pokes  on  its  back.  It  comes  out  every  three 
hours.  Some  people  don't  believe  in  it  but  just  in  case  it  is  real,  they're  staying  away 
from  the  water. 

e.  The  Lake  Champlain  monster  is  a  enormous  beast  about  20  feet  tall  and  3  feet  wide. 
It  is  very  ugly  and  gross. 


Comments:    The  first  three  types  of  responses  are  considered  satisfactory  to  varying 

degrees  because  they  describe  the  information  the  article  might  contain.  The 
student  answers  in  terms  of  genre  rather  than  content. 
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32 

27 

22 

15 

6 

12 

10 

6 

The  Lake  Champlain  Monster 

Reporting  Category:     External  Evaluation 
Grade  Level:  Eight  and  Twelve 

Response  Summary:  Frequency  (%) 

Gr8  Gr12 

a.  Detailed  evidence  on  both  sides  15  23 

b.  Vague  reference  that  two  sides  15  17 
will  be  presented 

c.  Evidence  of  one  side  presented 

d.  Completion  of  article 

e.  Irrelevant 

f.  Blank 


a.  I'd  expect  an  explanation  of  the  two  positions:  believers  and  skeptics.  There  would  be 
some  eyewitness  accounts  from  believers  and  some  really  cynical  remarks  from  the  skep- 
tics about  how  it's  all  a  publicity  stunt  to  get  people  to  come  and  spend  money  at  the 
lake.  Judging  from  the  title,  the  article  would  probably  end  with  some  sort  of  corny  re- 
statement of  the  first  two  sentences.  (Gr  12) 

b.  Choppy  sentences,  full  of  nonsense  facts  conjured  up  by  a  struggling  journalist  trying  to 
make  a  name  for  himself.  His  facts  appear  to  be  based  on  poor  conjecture  and  mere 
hearsay  of  town  folk,  as  can  be  seen  by  the  "cash  register"  sentence.  (Gr  12) 

c.  A  description  of  the  monster  should  be  in  the  article,  the  size  and  color,  why  it  likes 
warm  water  or  why  it  prefers  the  ringing  of  cash  registers.  (Gr  12) 

d.  The  monster  has  been  sighted  many  times  by  tourists  and  other  people.  (Gr  8) 

Skeptics  don't  believe  about  the  "Monster"  and  think  that  people  around  Lake 
Champlain  are  just  trying  to  make  a  buck.  But  the  people  around  Lake  Champlain 
strongly  believe  in  the  so-called  "Monster"  and  try  to  get  people  to  come  and  see  it.  (Gr 
8) 

e.  I  would  expect  information  concerning  the  economy  and  possible  economic  problems  in 
the  Lake  Champlain  territory.  The  monster  is  a  symbol  of  misfortune  and  a  struggling 
flow  of  money.  (Gr  12) 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


When  I  Read 


Based  on  what  the  poem  says,  describe  several  things  the  poet  might  do  when 
reading  a  novel. 


Reporting  Category: 
Grade  Level: 


Response  Summary: 


Internal  Evaluation 
Eight 


a.  Good  discussion  of  becoming 
engrossed 

b.  Good  discussion,  but  some 
misinterpretation 

c.  Literal  interpretation 

d.  Irrelevant/inappropriate 

e.  Blank/ 1  don't  know 


Frequency(%) 
36 

20 

19 

21 

6 


Examples: 

a.  The  poet  probably  reads  very  intensely.  She  probably  doesn't  stop  until  the  story  is 
over.  This  is  why  she  says  she  is  a  diver  and  doesn't  come  up  for  air  until  the  story  is 
over.  She  also  searches  for  a  meaning  or  moral  of  the  story  she  is  reading.  That  is 
why  she  says  she  is  a  tracker  and  rover. 

b.  She  reads  fast  because  in  the  first  two  sentences  she  talks  about  being  a  runner  and  a 
racer  and  lots  of  speed.  She  finishes  fast  with  time  to  spare  then  she  really  gets  into  it 
and  dives  right  in  and  doesn't  stop  until  she  finishes  the  book.  During  the  book  she 
looks  for  interesting  facts  and  information  to  get  more  into  the  book.  Then  she  might 
read  it  again  and  find  something  she  didn't  find  the  first  time  she  read  it. 

c.  The  poet  might  like  to  explore  the  book  to  find  out  every  little  item.  The  poet  might 
like  to  run  a  lot  and  relax  in  a  swimming  pool  or  something  while  reading  a  novel.  It 
might  sound  like  the  poet  would  go  camping  because  it  says  "I'm  an  explorer  and  I  al- 
ways find  something  I  didn't  know  was  there." 

d.  See,  instead  of  writing  I  plunge  right  in  and  until  the  story's  over  I  don't  come  up  for 
air.  Mrs  Morrison  should  of  wrote  "I  plunge  right  in  and  don't  come  up  for  air  till  the 
story  is  over."  I  think  it  goes  better  like  that.  The  way  she  had  it  didn't  make  sense. 
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Potatoes 


Appendix 


Reporting  Category:     Internal  Analysis 
Grade  Level:  Eight  and  Twelve 


Response  Summary: 


a.  Appropriate 

b.  Repeats  or  summarizes  story 

c.  Irrelevant,  inappropriate 


d.  Blank /I  don't  know 


Frequency  (%) 

Gr8 

Gr12 

47 

73 

19 

17 

25 

7 

9 

3 

Examples  (Grade  8): 

a.  Antoine  Parmentier  understood  that  if  you  take  something  (even  potatoes)out  of  man's 
reach,  he  will  turn  all  efforts  towards  achieving  that  one  thing  —  i.e.,  put  a  guard  on  a 
potato  field  and  potatoes  become  very  valuable  all  of  the  sudden. 

The  people  thought  it  must  have  been  something  really  good  and  expensive.  But  it  was 
just  potatoes. 

He  knew  that  people  were  curious  and  would  want  to  see  what  the  soldiers  were  guard- 
ing. 

b.  He  had  planted  potatoes  in  a  garden  and  a  lot  of  guards  watched  the  garden.  When  the 
guards  left  they  stole  the  potatoes.  The  reason  why  they  did  this  is  because  that's  the 
only  way  they  would  eat  the  potatoes. 

c.  He  knew  if  they  thought  they  were  good  they  would  eat  them. 

If  you  have  almost  nothing,  then  you  should  try  it.  Or  try  it  you  might  like  it. 

He  shows  that  humans  are  just  like  animals  to  and  have  to  eat  but  are  at  a  higher  intel- 
ligence level. 
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On  Their  Own:  Student  Response  to  Open-Ended  Tests  in  Reading 


The  Cave  /  The  Dreamer 


Which  boy  seems  to  be  more  unhappy?  Explain  the  reasons  for  your  choice. 

Reporting  Category:        Internal  Evaluation 
Grade  Level:  Eight 


Response  Summary: 

a.  Gives  reason,  referring  to  either  boy's  feelings,  unpleasant  surroundings  and/or 
large  amount  of  time  spent. 

The  boy  in  "The  Dreamer"  seems  more  unhappy  because  the  poem  states  the  reason  for 
the  boy's  troubles  and  emphasizes  his  gladness  to  "get  away  from  it  all!" 

The  dreamer  seems  to  be  more  unhappy  because  it  says  as  though  God  was  to  say: 
"Even  at  your  age,  you  could  do  better."  He  seems  as  though  everyone  hates  him, 
probably  even  God. 

The  boy  in  the  poem  The  Dreamer  seems  to  be  more  unhappy.  He  seems  to  be  more  un- 
happy because  he  thinks  he  can  do  better  and  that  makes  him  very  unhappy. 

The  boy  who  went  to  the  cave.  Because  he  felt  lonesome  and  would  just  sit  in  the  cave 
doing  nothing. 


b.  Compares  the  two  poems: 

The  dreamer  was  more  unhappy  because  in  "The  Cave"  it  says  that  sometimes  when  the 
boy  was  troubled  he'd  go  to  a  little  cave,  meaning  he  wasn't  troubled  all  the  time.  But  in 
"The  Dreamer"  it  says  he  :>pent  his  childhood  hours  in  a  den,  meaning  he  was  unhappy 
most  of  the  time. 


c.  Gives  generalized  reasons: 

The  Dreamer  seems  more  unhappy  because  he  explains  it  in  an  unhappy  way. 

d.  Misunderstands  poem: 

The  Dreamer  because  he  has  to  stay  in  all  the  time  instead  of  maybe  going  out. 

The  boy  in  The  Dreamer  because  he  spent  his  childhood  in  a  den  of  rushes  watching  the 
gray  rain  braille  the  surface  of  the  river. 
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